Estimates of constrained multi-class a posteriori probabilities in time series problems with neural - Neural Networks, 1999. IJCNN '99. International Joint Conference on

In time series problems, where time ordering is a crucial issue, the use of Partial Liklihood Estimation (PLE) represents a specially suitable method for the estimation of parameters in the model. We propose a new general supervised neural network algorithm, Joint Network and Data Density Estimation (XWDE), that employs PLE to approximate conditional probability density finctions for multi-class classification problems. The logistic regression analysis is generalized to multiple class problems with sofhnm regression neural network used to model the aposteriori probabilities such that they are approximated by the network outputs. Constraints to the network architecture, as well m to the model of data, are imposed resulting in both a jlaible network architecture and distribution modeling. We consider application of JNDDE to channel equalization andpresent simulation results. Introduction where w represents the network parameters, v the model of data over-parameters, J the number of classes, N the number of samples, Cij the class label at time instant i for classj, the observed history vector at time i. Function f ( ) represents the a posteriori probabilities and i7J the indicator index of classj at time i. There are several approaches to estimation of the aposteriori class probabilities. One is based on Strict Sense Bayessian classifiers (SSB). Following [3], we call a classifier Strict Sense Bayesian (SSB) if its outputs are estimates of the a posteriori probabilities of the classes. In a similar way, for a training viewpoint, we call a cost h c t i o n is SSB if it is minimized when the classifier is SSB. In [3] and [4], a SSB cost hction is defined such that it has a unique minimum when output y coincides with a posteriori class probabilities, which we are trying to determine. Necessary and sufficient conditions for a cost to be SSB are also given in the reference. We propose softmax regression for the a posteriori class probabilities, partial likelihood is a recent extension of maximum likelihood introduced by Cox [l]. It provides a partiduly suitable fomdation for time series problems and a partial likelihood formulation for real-time signal be written as follows: f,(Ci,j lxi,wj&) (2) and Gaussian m i w e model for observation, processing is given in [2]. The maximum PLE (MPLE) can fm(zi m..rk' . m d , . 62 m (3) 0-7803-5529-6/99/$10.00 01999 IEEE where the dimensionality of both models does not need to be the same. OPDE-MPL learning rule is obtained for the Generalized Softmm Perceptron (GSP) universal classifier (1) architecture, see Fig.1.